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| /' | | We study the rank of complex sparse matrices in which the supports of different columns 

have small intersections. The rank of these matrices, called design matrices, was the focus 
of a recent work by Barak et. al. [BDWYfl , in which they were used to answer questions 
regarding point configurations. In this work we derive near-optimal rank bounds for these 
matrices and use them to obtain asymptotically tight bounds in many of the geometric 
applications. As a consequence of our improved analysis, we also obtain a new, linear 
algebraic, proof of Kelly's theorem, which is the complex analog of the Sylvester-Gallai 
theorem. 



1 Introduction 

The classical Sylvester-Gallai (SG) Theorem states the following: Given any finite set of points 
fT) . in the Euclidean plane, not all on the same line, there exists a line passing through exactly 

m 
O 



two of the points. This result was first conjectured by Sylvester in 1893 Syl93| and then 
proved independently by Melchior in 1940 |Mel40] and Gallai in 1943 (in an answer to the 
same question independently posed by Erdos [Erd43] ). The following complex variant of the 
£Sj | Sylvester-Gallai Thoerem was proved by Kelly [Kel86j in response to a question of Serre: 

Given any finite set of points in C d , not all on the same complex two-dimensional plane, 
there exists a line passing through exactly two of the points. The above result is tight over 
the complex numbers, since there are two-dimensional configurations of points satisfying the 
condition on triples. The survey by Borwein and Moser [BM90J gives a very good overview 
of the SG Theorem, its proofs and the different variants that have been studied in the past. 
One application of our techniques, discussed later, is a new proof of Kelly's theorem, which is 
significantly simpler than Kelly's original proof and is very different than the recent elementary 
proof of [EPS06j . 

In recent years, variants of the SG theorem have been useful in studying certain structural 
questions arising in theoretical computer science. Variants of the SG theorem were useful in 
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understanding the structure of low-depth arithmetic circuits [DS061 [KS09, SS10]. Quantitative 
versions of the SG theorem were shown to be closely linked to the structure of linear Locally 
Correctable Codes [BD WYlTj IBDSSllj . These applications join a growing number of papers 
in which geometric theorems regarding point/line arrangements are finding applications in 
theoretical computer science. We refer the reader to the recent survey [Dvil2| for an overview 
of some of these applications. 



1.1 Rank of design matrices 

Motivated by the application to point configurations, [BDWYllJ studied the rank of certain 
matrices, called design matrices, and proved lower bounds on their rank. These bounds were 
then used to prove quantitative analogs of the SG theorem. We begin by defining design 
matrices formally. For a vector v G F n , where F is a field, we denote by supp(-u) = {i G 

MK^o}. 

Definition 1.1 (Design matrix). Let A be an mxn matrix over a field F. Let R±, . . . , R m G F n 

be the rows of A and let Ci, . . . , C n G F m be the columns of A. We say that A is a (q, k, t)-design 
matrix if 

1. For all i G [m], \supp(Ri)\ < q. 

2. For all j G [n], \supp(Cj)\ > k. 

3. For all j\ / J2 G [n], \supp{Cj L ) n supp(Cj 2 )\ < t. 

A 'typical' setting of the parameters that often arises in applications is when q is a small 
constant, m ~ n 2 , k ~ n and t is a constant. The main result in [BDWYli] is the following 
rank bound. 

Theorem 1.2 ([BD WY11| ). Let A by an m x n matrix. If A is a (q,k,t) design matrix then 

\k 

For the aforementioned 'typical' setting of the parameters, one get a lower bound of n — 
0((n/k) 2 ) on the rank. By improving a key lemma in [BDWY11] using a more careful analysis, 
we are able to prove the following new bound. 

Theorem 1.3. Let A by an m x n matrix. If A is a (q,k,t) design matrix then 

, , .. n mtqiq — 1) 

r*nk{A)> - ._ 1W >n 



i , q(q-l)mt &2 

As a corollary, we get: 
Corollary 1.4. Let A by an m x n matrix. If A is a (q,k,t) design matrix then 

, , n ntqiq — 1) 
rank(A) > — - > n ^ '-. 
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Proof. If m < nk then we are done (substitute m = nk into the bound in Theorem ll.3p . 
Otherwise, we can remove rows of A until we are left with a new matrix A' that has exactly 
w! = nk rows and s.t A' is also a (q, k, i)-design matrix (as long as m > nk there has to be 
a row we can remove and maintain that each column has at least k non zeros). Now, apply 
Theorem 11.31 on A' and use the fact that rank(^4) > rank(-A'). □ 

Here, for the 'typical' setting, we get a rank bound of n — 0(n/k), which is asymptotically 
better then the one obtained in Theorem 11.21 

We also prove a variant of Theorem 11.31 in which q does not appear. Note that if each 
column of A has support of size exactly k, then the total number of nonzero entries in A is nk. 
Thus the average size of the support of a row would be nk/m. In general it can be shown that 
replacing q with nk/m in Theorem 11.31 would give a false statement. However, we show that 
we can replace q with 0(nt/k). This is exactly the average row-support when m = 0(k 2 /t) 
and, in this regime, the bound on the rank (which is now independent of q) is tigh10. 

Theorem 1.5. Let A by an m x n matrix. If A is a (q,k,t) design matrix with k < nt then 

6mn 2 t 3 

rank(A) >n — . 

As before, by replacing m with nk we get the following corollary. 
Corollary 1.6. Let A by an m x n matrix. If A is a (q, k,t) design matrix with k < nt then 

6n 3 i 3 

rank(A) > n p— . 



Related work: The problem of bounding the rank of matrices with certain patterns of zeros 
and non-zeros is not new, and has been studied in the past in a variety of contexts. One line of 
research comes from Hamada's conjecture |Ham73t lJT09, BIW07]. In this setting however, the 
notion of being a design is stricter than the notion we use in this paper. Another line of research 
that studies the zero-nonzero patterns of matrices and their ranks has many applications to 
graph theory [FR07|. Rank bounds on matrices with 'sign patterns' of positive and negative 
entries have also received a great deal of attention in recent years [RS08|. IFor02| , and it would 
be interesting to see if our techniques can say anything meaningful in this setting. 



1.2 Square design matrices and monotone rigidity 

Theorem 11.51 which removes the dependence on q, allows us to get meaningful lower bounds 
on the rank of square design matrices. The results of [B DWYlT] did not give anything for such 
matrices. Let A be an n x n matrix such that every column has support of size k ~ \fn and 
such that for every two columns, the size of the intersection of the supports of the two columns 
t is O(l). For instance, the zero-nonzero pattern of the projective plane incidence matrix has 

1 Observe that for an m x n matrix that is a (q, k,t) design, if n = Q(k/i), then m = Q(k 2 /t). This follows 
by a simple inclusion- exclusion argument on the union of the supports of any Q(k/t) columns of A. 



3 



this structure. In this case we can obtain a bound of Q(n) on the rank of such a matrix - not 
by applying our rank bound directly but by applying Theorem II. 51 to the matrix after deleting 
a linear number of columns. 

A simple consequence of this result, proved in Section |4j is that, if one takes the n x n 
incidence matrix of the projective plane and changes a small number of l's in the matrix to 
arbitrary values, then the resulting matrix has high (linear in n) rank. This can be viewed 
as a restricted model of the matrix rigidity problem of Valiant [Val77]. A matrix is rigid if 
changing a small number of its positions cannot decrease its rank by much. Valiant showed 
that a linear circuit computing a transformation given by a rigid matrix cannot have linear 
size and logarithmic depth. Hence, the problem of finding an explicit rigid matrix will imply 
circuit lower bounds that are beyond our reach at this point. Our restricted model allows one 
to only change positions in the matrix that are non-zero. Even though this result does not 
yield any interesting result on circuit lower bounds, we find it encouraging in that it gives a 
way to control rank under some type of perturbations. The full details are given in Section [H 

1.3 Configurations with many collinear triples 

Given a set of points vi,...,v n G C d , we call a line that passes through exactly two of the 
points of the set an ordinary line. A line passing through at least three points is called special. 
We will use dim(t>i, . . . , v n ) to denote the dimension of the linear span of v\, . . . ,v n and by 
affine-dim(t>i, . . . , v n ) the dimension of the affine span of v\, . . . ,v n (i.e., the minimum r such 
that v\, . . . , v n are contained in a shift of a linear subspace of dimension r). 

The main geometric application studied in [BD WYlT] was to extend the SG theorem to 
configurations of points termed 5-SG configurations. 

Definition 1.7 (<5-SG configuration). We say that a set of points v\,V2, ■ ■ ■ ,v n G C d is a 5-SG 

configuration if for every V{, i G [n], at least 5(n — 1) of the remaining points lie on special 
lines through Vi. 

Setting 5 = 1 we can state the original SG theorem as saying that any 1-SG configuration 
vi,...,v n (over M) has affine-dim(t>i, . . . ,v n ) < 1 (i.e., is contained in a line). Kelly's theorem 
[Kel86| obtains the bound affine-dim(i>i, . . . , v n ) < 2 for 1-SG configurations over C (the bound 
2 is tight in this case). The following theorem, proved in [BDWY11] gives a generalization of 
Kelly's theorem to 5-SG configurations. 

Theorem 1.8 f |BDWYll| ^. Let V = {vt, . . . , v n } C C d be a 5-SG configuration. Then 

affine- dim(v\, . . . , v n ) < 13/5 2 . 

When 5=1 one gets a bound of 9 on the affine dimension. 

There are two shortcomings of this theorem. The first is the quadratic dependence on 5. 
Placing the points on 1/5 lines in general position one can construct a 5-SG configuration with 
dimension 0,(1/5). It was left as an open question in [ BDWY11| to close this quadratic gap 
between the lower and upper bound on the dimension of 5-SG configurations. The second issue 
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is that one does not recover Kelly's theorem from the proof of [BDWYiT] , but only an inferior 
bound of 9 on the dimension. We are able to correct both of these issues. 

Theorem 1.9. Let V = {vi, . . . ,v n } C C d be a 5-SG configuration. Then 

affine-dim(vi, . . . ,v n ) < 12/5. 

When 5 = 1 one gets a bound of 2 on the affine dimension. 

There are two known proofs of Kelly's theorem. Kelly's original proof, answering a question 
by Serre, used deep results from algebraic geometry. An elementary proof was recently found 
by Elkies, Pretorius and Swanepoel [EPS06J. Our proof is conceptually very different from 
both of these and uses only elementary linear algebra. 

Theorem ll.9l is proved, as in [BDWYiT] . by reduction to the rank bound for design matrices. 
One constructs a design matrix whose co-rank bounds the dimension of the configuration and 
then applies one of the bounds on the rank of these matrices. 

Average-case version: A natural variant on the definition of a 5-SG configuration is to 
only require the presence of many pairs of points on special lines (instead of requiring each 
point to belong to many such pairs). In [B DWY11] it was shown that: 

Theorem 1.10 ( [BDWYiT] ). Let V = {v\, . . . ,v n } C C d be a set of n points such that at least 
5n 2 (unordered) pairs of them lie on special lines. Then there exists a subset V C V such that 
\y'\ > (5/6)n and so that V' is a 5-SG configuration. 

Combining this result with our improved bound on the dimension of 5-SG configurations 
we get the following improvement to a theorem from [BDWYiT] . 

Corollary 1.11. Let V = {v\, . . . ,v n } C C d be a set of n distinct points. Suppose that there 
are at least 5n 2 unordered pairs of points in V that lie on a special line. Then there exists a 
subset V C V such that \V'\ > (5/6)n and affine- dim(V) < 0{l/5). 

1.4 Flats of higher dimension 

A k-flat is an affine subspace of dimension k. Let fl.(«i, . . . , Ufe) denote the flat spanned by these 
k points (it can have dimension at most k — 1). We call vi,... ,x>k independent if their flat is 
of dimension k — 1, and say that vi, . . . ,Vk are dependent otherwise. Considering some fixed 
finite subset V C C d of size n we call a fc-flat ordinary if its intersection with V is contained in 
the union of a (k — l)-flat and a single point (this agrees with the definition of an ordinary line 
when k = 1). A fe-flat is elementary if its intersection with V has exactly k + 1 points. Notice 
that for k = 1 (when flats are lines) the two notions of ordinary and elementary coincide. 

The next definition generalizes the notion of a 5-SG configuration for higher dimensional 
flats in two different ways (using ordinary/elementary flats). For k = 1 both definitions 
coincide. 
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Definition 1.12 (5-SGk, <5-SG£). The set V is a 5-SGf, configuration if for every independent 
vi, . . . ,Vk E V there are at least 5n points u E V such that either u E fl(vi, ... ,Vk) or the 
k-flat fl(vx, ... ,Vk,u) contains a point w outside fl(v\, . . . ,Vk) U {u}. The set V is a 5-SGk 
configuration if for every independent v\ , . . . , Vk E V there are at least 8n points u E V such 
that either u E fl{v\, ■ ■ ■ , Vk) or the k-flat fl(v±, . . . , v k , u) is not elementary. Notice that a 
5-SG* k configuration is also a 5-SGk configuration and that for k = 1 both are the same. 

In [BD WYTT] . the following theorem was proved. 

Theorem 1.13 ([B DWYllj ). Let V and V* be a 5-SGk and a 5-SGf. configurations respectively 
in C d . Then: 

1. affine-dim{V*) < O {{k/5) 2 ). 

2. affine-dimiV) < 2 ck /5 2 , where C > is a universal constant. 

Prior to this result, the only known bound for configurations with many special A;- flats 
was a result proved by Hansen and Bonnice-Edelstein |Han65l IBE67] which gives a bound of 
0(k) on the dimension of a 1-SG& (or 1-SG£) configuration over the reals. Since Theorem II .131 
is proved in a black-box manner using the result for <5-SG configurations we can plug-in our 
improvement, given in Theorem 11.91 to obtain the following. 

Theorem 1.14. Let V and V* be a 5-SGk and a 5-SG* k configurations respectively in C d . 
Then: 

1. affine-dim{V*) <0(k/5). 

2. affine-dim(V) < C k /5, where C > is a universal constant. 

Notice that, whereas the improvement for (5-SG£ configurations is only quadratic, the im- 
provement for 5 — SGk configurations is exponential (this is due to the way the basic bound 
is amplified in the induction on k). The proof of this theorem is identical to the proof of 
Theorem 11.131 appearing in [BD WYllj . only with Theorem 11.81 replaced by Theorem 11.91 For 
completeness, we give the details in Section 15.21 

1.5 A variation on Preiman's Lemma 

Consider a finite set A in some abelian group. One can define the sumset A + A = {a± + 
«2 | a-i , a2 E A} in a natural way. A well-known result in additive combinatorics is the following 
lemma, known as Freiman's lemma, which derives structural information on A, given bounds 
on the size of A + A. 

Lemma 1.15 (Freiman's lemma. See |TV06| ). Let A be a finite subset of M. d and suppose 
\A + A\ < K\A\. Then A is contained in a linear subspace of dimension at most [K — lj . 

Clearly, the condition |^4+j4| < K\A\ can be replaced by |{(oi+02)/2| a\,ai E A\\ < K\A\, 
where we replace sums with mid-points. Surprisingly enough, the original proof of this lemma 



6 



works also when we replace mid points with any point on the line segment connecting the two 
points. More formally, for two sets A, B C M d and any function / : A x B \— > M. d we can 
define A+jB = {f(a,b)\a 6 A, b € B}. Then, as long as f(a,b) is on the line segment 
connecting a, b (and is different from a, b) we get the same conclusion as in Lemma 11.151 
assuming < if|^4|. 

Intuitively, our results for <5-SG configurations are of a similar flavor since the assumption 
of Freiman's Lemma (in its generalized form just stated) implies the existence of many pairs 
of points on special lines. We are able to use our techniques to derive the following theorem 
(whose proof appears in Section [6]). 

Theorem 1.16. Let A be a finite subset of and let f : A x A i— )■ C d be any function 
such that for all a± ^ 02 G A we have f (01,02) = aa% + (1 — 0)02 for some a = 0(01,02) : 
Ax A 1— >£ C \ {0, 1} (i.e., f(a\, 02) is on the line passing through a\, a-i minus the two points 
01,02). Suppose that 

\A+ f A\ <K -A. 

Then dim(A) < 0(K 2 ) 

This theorem relaxes the conditions of Freiman's lemma by allowing (a) points in complex 
space and (b) the value of / (01,02) to be outside the convex hull of 01,02- On the other hand, 
we get a worse bound of 0(K 2 ) instead of O(K). We do not know if this quadratic loss is 
needed or not. 

1.6 Organization 

In Section [2] we introduce some preliminaries related to the technique of matrix scaling. In 
Section [3] we prove Theorem 1 1 . 3 1 and Theorem 11.51 The rank bound for square design matrices 
and the application for monotone linear circuits is given in Section [H In Section [5] we prove 
our main application, Theorem II. 9i In Section [5.21 we prove the high dimensional variant. In 
Section [6] we prove Theorem 11.161 

2 Preliminaries — Matrix Scaling 

One of the most important ingredients in the proof of the rank bound for design matrices is 
the notion of matrix scaling. Informally, the matrix scaling theorem states that if a matrix 
does not have any large zero sub- matrices, then one can multiply the rows and columns of the 
matrix by non-zero scalars so that all the row sums are equal and all the column sums are 
equal (assuming the entries are non- negative) . 

The technique of matrix scaling originated in a paper of Sinkhorn [Sin64j and has been 
widely studied since then (see [LSWOO] for more background). It was used in [BDWYll] for 
the first time to study design matrices, and we build upon their work and extend it. 

We first set up some notation. For a complex matrix X, we let X* denote the matrix X 
conjugated and transposed. Also we let Xij denote the (i,j) entry of X. For two complex 
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vectors u, v £ C m , we denote their inner product by (u, v) = YliLi u i ' v j an d ^ ll n ll = y ( u -> u ) 
denote the £2 norm of the complex vector u. 

Definition 2.1. [Matrix scaling] Let A be an m x n complex matrix. Let p G C m ,7 G C n be 
two complex vectors with all entries non-zero. We denote by 

SC(Ap,7) 

the matrix obtained from A by multiplying the (i,j)'th element of A by pi ■ jj. We say that 
two matrices A, B of the same dimensions are a scaling of each other if there exist non-zero 
vectors p,j such that B = SC(A, p, 7). It is easy to check that this is an equivalence relation. 
We refer to the elements of the vector p as the row scaling coefficients and to the elements of 
7 as the column scaling coefficients. Notice that two matrices which are a scaling of each other 
have the same rank and the same pattern of zero and non-zero entries. 

Below we define a property of matrices that gives sufficient conditions for finding a scaling 
of a matrix which has certain row and column sums. 

Definition 2.2 (Property-,!?). Let A be an m x n matrix over some field. We say that A 
satisfies Property-,!? if for every zero sub-matrix of A of size a x b it holds that 

m n 

For example, a square matrix has Property-S if is has a non-zero generalized diagonal. 
Also, notice that this property is maintained under concatenation (say, putting two matrices 
with the same number of columns one under the other). The following theorem is the main 
tool we will use. Its proof uses ideas from convex optimization but we will only need to use it 
as a black box. 

Theorem 2.3 (Matrix scaling theorem, Theorem 3 in [RS89J ). Let A be an mxn real matrix 
with non-negative entries which satisfies Property-S . Then, for every e > ; there exists a 
scaling A' of A such that the sum of each row of A' is at most 1 + e and the sum of each 
column of A' is at least m/n — e. Moreover, the scaling coefficients used to obtain A' are all 
positive real numbers. 

In our proof will use the following easy corollary of the above theorem that appeared 
in [BDWYf I]. This corollary is obtained by applying the matrix scaling theorem to the matrix 
obtained by squaring all entries of the original matrix. 

Corollary 2.4 (Corollary from [BD WYlT] ). Let A = (aij) be an mx n complex matrix which 
satisfies Property-S . Then, for every e > 0, there exists a scaling A' of A such that for every 
i 6 [m] 

je[n] 

and for every j G [n] 



id\m 



a , , r > m/n 



S 



3 Proof of the rank bound 



In this section we will present the proofs for Theorem 11.31 and Theorem 11.51 The proof follows 
the same general outline as the one appearing in [BDWYlTj : 

Step 1 — Scaling: Given the design matrix A, we construct a scaling A' of A where every 
column has large £2 norm and every row has small £2 norm. Since A need not satisfy Property- 
S, we are not be able to apply Corollary 12.41 directly. Instead we first find a matrix B whose 
rows are chosen from the rows of A, with repetitions, such that no row is chosen too many 
times. If each row of A occurs a maximum of c times in B, then we call B a c-cover of A. We 
can then apply Corollary 12.41 to get a scaling of B with equal row norms and equal column 
norms, and then use the scaling of B to derive a scaling of A with the desired properties. 
In [BDWYlTj B is taken to be a g-cover of A. 

Step 2 — Obtaining a diagonal dominant matrix: Given the scaling A' of A we consider 
the matrix M = A'* A'. Clearly all the diagonal entries of M, which correspond to the squared 
£2 norms of the columns of A' , are large. We use the design properties of A, as well as the 
properties of the scaling to show that the sum of square of the off-diagonal entries of M is 
small. Matrices such as M are called 'diagonal dominant' and bounding their rank can be 
done in various ways (see e.g., [Alo09] ) . In this step, our calculation gives a tighter analysis 
of the bounds of the entries of M and we are hence able to obtain the stronger rank bounds 
compared to [BDWYllj . 

3.1 Covering lemmas 

The following lemma is implicit in [BDWYTTj and shows how one can find a cover of a matrix 
A that satisfies Property-.?. Recall that a matrix B is a c-cover of A if each row of B is a row 
of A and each row of A appears at most c times in B. 

Lemma 3.1. Let A by an m x n matrix over C that is a (q,k,t) design matrix. Then there 
exists an nk x n matrix B that is a q-cover of A, and such that B satisfies Property-S . 

Proof sketch. B is constructed as follows: for each i £ [n], we let B{ be a k x n submatrix of 
A which has no zeros in the i'th column. Let B be the nk x n matrix which is composed of 
the concatenation all matrices Bi,i £[n]. □ 

We now prove a variant of Lemma 13.11 where every row of A appears at most 6nt/k times 
in B. In some settings 6nt/k might be smaller than q and then this variant might give a 
potentially stronger rank bound (as stated in Theorem II. 5h . Observe that this rank bound is 
independent of the parameter q. 

Lemma 3.2. Let A by an mx n matrix over C that is a (q, k, t) design. Suppose k < nt, then 
there exists an nk x n matrix B that is a 6nt/k-cover of A, and that satisfies Property-S . 
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Proof. We split the set of n columns of A into I sets, where I < \2nt/k~\, each of size at most 
k/2t. Call these sets Si, S%, . . . , S(_. For each Si, we will first construct a matrix Bi which is 
an (\Si\ ■ k) x n matrix that will be composed of the rows of A, where each row appears at 
most 2 times. The matrix B will be an nk x n matrix which is composed of all the matrices 
Bi, 1 < i < I. In other words the set of rows of B is the multi-set obtained by taking all the 
rows of all the B{. 

The matrix Bi is constructed as follows. For each column in Si, there are at least k/2 rows 
such that none of the other columns in Si has a nonzero entry in that row. This is because the 
intersection of the support of any two columns has size at most t, and there are at most k/2t 
columns in Si. Thus the support of each column can intersect the union of the support of all 
other columns in Si in at most k/2 locations. For each column of Si, pick some k/2 rows such 
that none of the other columns in Si has support which intersects that row, and add 2 copies 
of each of those rows to the matrix Bi. Do this for each column in Si- It follows immediately 
from construction that each row of A appears at most two times in each Bi. 

Since each row of A appears at most two times in each Bi, and B is composed of < \2nt/k~\ 
such matrices Bi, thus each row of A appears at most 2\2nt/k~\ < Ant/k + 2 < Qnt/k times in 
B (using the bound k < nt). 

To see that B satisfies property-S, observe that B can be written as the union of k square 
nxn matrices each with nonzero entries on the main diagonal. For each of the nxn matrices, 
we would take \Si\ rows per set Si, where each row corresponds to one of the columns in Si, 
such that none of the other columns in Si have a nonzero entry in that row. □ 

The next lemma shows the one can use a cover of A to find a good scaling of A. This 
lemma is also implicit in [BD WYlTj and we give the proof sketch only for completeness. 

Lemma 3.3. Let A by an mxn matrix over C, and let B be an nkxn matrix that is a c-cover 
of A, and such that B satisfies Property-S . Then, for every e > 0, there exists a scaling A' of 
A in which each row of A' has £2 norm at most yl + e and each column of A' has £2 norm at 
least y/(k — e)/c. 

Proof sketch. Fix e > and apply Corollary 12.41 on B to obtain a scaling B' of B such that 
the £2 norm of each column is at least y/k — e, and the £2 norm of each row is at most \/I + e. 
We now use this scaling B' of B to obtain the scaling A' of A. The scaling of the columns 
used to get A 1 is the same as the scaling coefficients for the columns of B'. We pick the scaling 
coefficients of the rows of A' as follows: for each row R that appears in A, we look at the 
occurrences Ri, R2, ■ ■ ■ , Ri of the same row in B and look at the scaling coefficients for those 
rows in B' . Say the coefficients are s\, s^, ■ ■ ■ , Sj (i < c), then we take maxjsi, . . . s^} to be the 
scaling coefficient of row R. If the row R does not appear in B at all, then we pick the scaling 
coefficient to be such that the final £2 norm of the row is 1. One can easily verify that A' is a 
scaling of A with each row of A' having £2 norm at most \f\ + e and each column having £2 
norm at least ^/ {k — e)/c. □ 
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3.2 Proof of Theorem QH and Theorem [T5I 

Before proving the theorems we prove two more lemmas. The first lemma is the main new 
ingredient in our proof enables us to get a tighter bound on the entries of the diagonal dominant 
matrix M (see proof outline above). 

Lemma 3.4. Let A be an m x n matrix over C. Suppose that each row of A has £2 norm < a 
and suppose that the supports of every two columns of A intersect in at most t locations. Let 
M = A* A. Then 

E l M *il 2 - tmaA - 

i¥=3 

Moreover if we know that the size of the support of every row in A is at most q, then 



^2\M t ,\ 2 < (l-^j tma 4 



Proof. For 1 < % < n, Let Cj denote the ith column of A. Then 



E 



E A ^ 



k=l 



ijLj k=l 



< 



m In 

'E El' 1 

k=l \i=l 



hi 



< tma . 



When there are at most q nonzero entries per row, we have 

\ 2 



m In 



m in 



t E E i^i 2 i%i 2 = t E E i^i 2 -'E E i^i 4 



k=i \i=i 

m / n 



k=l \i=l 



-. \ m / n 

'EEw 



< L 7 I > \J± ki 

k=l \i=l 

\ '"' / " 
1 I t ? I 7 \Jiki 

fc=l \i=l 



k=l \i=l 

\ 2 
2 I 



I 1 - - ) tma 11 . 



□ 



11 



The second lemma is a variant of a folklore lemma on the rank of diagonal dominant 
matrices (see, e.g., [Alo09] ) . 

Lemma 3.5. Let M be an n x n Hermitian matrix such that for each i G [n], Ma > L, where 
L is some positive real number. Then, 

rank(M) > 



nl? + £ ¥j W 



Proof. First, note that, w.l.o.g, we can assume Mn = L for all i € [n]. If not, we can replace 
M with a scalinj 

most 1 we have 



M with a scaling M' of M defined as = , L ■ M»a. Since all scaling coefficients are at 



EKi 2 <Ei^i 2 

and both matrices M and M' have the same rank. To bound the rank of M (assuming all 
diagonal elements are equal to L) we denote its (real) non-zero eigenvalues by Ai, A2, • • • , A r , 
where r = rank(M). Then 

n 2 L 2 = tr(M) 2 = (j>^) 

r n 

t=i tj=i 

nL 2 + J2\Mij\ 2 

Rearranging, we get the required bound on r. □ 

Proof of Theorem \1.3[ Let A be an (q,k,t) design matrix and fix some e > 0. Using Lem- 
mas 13.11 and 13.31 we obtain a scaling A' of A where each row of A' has £2 norm at most yl + e 
and each column has £0 norm at least y/(k - e)/q. Let M = A'* A'. Then Mu > (k— e)/q and, 

by Lemma \3^\ Yli^j \Mij\ 2 < ^1 — |J tm(l + e) 2 . Applying Lemma [331 to M we get that 

n 2 f ^ 2 

rank(M) > g 

n(^) +(l-i)kll + r)2 

Taking e to zero and simplifying, we get 



rank(M) > " w > n - mtg(g - l)//c 2 



1 + 



l)mt 



where the second inequality follows from the fact that 1/(1 + x) > 1 — x for all x. Since 
rank(A) = rank(A') > rank(M), Theorem [L3] follows. □ 
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Proof of Theorem \1.5[ The only change in the proof of Theorem ll.5l is that instead of Lemma l3.ll 
we use Lemma [3. 21 By Lemmas 13. H and 13.31 we get a scaling A' of A where each row of A' has £2 
norm at most y/1 + e and each column has £2 norm at least y/(k - e)k/6nt. Letting M = A'* A' 

as before, we have Ma >{k- e)k/6nt and, by LemmaE31 Ei^j \ M ij? < (l - |) tm(l + e) 2 . 
Applying Lemma [331 as before we get 

2 3 / 7 4 

r > > n — 6mn t k . 

□ 



Tight examples for Theorem II. 3t One might hope that the bound of tm in the The- 
orem 11.31 (where t is the maximum intersection of any two columns) can be replaced by im, 
where i is some kind of average of intersections of all pairs of columns. One attempt towards 
showing such a statement would be by introducing another Cauchy-Schwarz after the one used 
in the proof of Lemma 13.41 However, this does not seem to help. The resulting bound after 
the second Cauchy-Schwarz seems to give a bound of yjYli^j ^% m i which is worse than tm. 
To see why max t should not be replaceable with average t in the final statement of the rank 
bound is by the example of a square n x n matrix A with n/s blocks of size s x s arranged 
along the diagonal. These blocks have all entries equal to 1, and elsewhere. Then the rank 
of this matrix is n/s. Now, max t equals s. If n is much larger than s, then average t is much 
smaller than 1. Now take this matrix A and randomly pick n/100 columns of A and call this 
new matrix A'. Then we can still have that average t < 1, k = s and q = s/100. Plugging 
it into our rank bound for design matrices (with max t replaced by average t) will give us a 
lower bound of Q(n) on the rank of A' , whereas we know that rank of A' is at most n/s. 

4 Rank bound for square design matrices 

We start by deriving an easy corollary that bounds the rank of square design matrices. 

Theorem 4.1. Let A be an n x n matrix that is a (q,k,t) design matrix. Then rank(A) = 
n(k A /nt 3 ). 

Proof. Delete any n — k 4 /Wnt 3 columns of A to get a new matrix A' . Then A' is a n x k 4 /Wnt 3 
matrix that is also a (q,k,t) design. Applying Theorem 11.51 to A', and doing the calculation, 
we get that rank(^') > k 4 /10nt 3 - k 4 /20nt 3 ). Thus rank(A) = Q(k 4 /nt 3 ). □ 

Note that the incidence matrix of the projective plane of order p is a (p 2 +p+ 1) x (p 2 +p+ 1) 
matrix which is a (p,p, 1) design. Thus any square matrix that has the same zero-nonzero 
pattern as the incidence matrix of the projective plane must have linear rank. Such a result 
was known for the — 1 valued incidence matrix of the projective plane matrix using the 
argument by Alon [Alo09]. However the result above allows us to get a bound on the rank 
with just the information of the zero-nonzero pattern. In contrast, over finite fields, the rank 
of the projective plane incidence matrix is sub-linear in n. 
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4.1 Monotone rigidity 



One of the motivation for proving rank bounds for matrices, using only information on their 
support, comes from a longstanding open problem in complexity theory, known as matrix 
rigidity. Informally, a matrix is rigid if one cannot reduce its rank by much by changing a 
small number of its entries in each column. More formally, we have: 

Definition 4.2 (Matrix rigidity |Val77j ). Let A be an n x n matrix over some field. We say 
that A is (r, s) -rigid if A cannot be written as A = L + S with 

1. L a matrix of rank at most r and 

2. S a matrix with at most s non zeros per column. 

In |Val77] . Valiant showed that if A is (n a , cm)-rigid (for some constant a > 0) than a 
linear circuit (a circuit with fan-in 2 gates, each computing a linear combination of previously 
computed gates) computing the mapping x \— > x t A cannot have both 0(n) size and <3(log(n)) 
depth. Since proving such lower bounds is beyond the reach of current techniques, constructing 
explicit rigid matrix (over any field) has become a much sought after goal. We refer the reader 
to the survey [Lok 09] for more background on this longstanding open problem. 

Using our results on design matrices, we can construct an explicit matrix that is highly 
rigid, as long as one only changes its non-zero entries. 

Definition 4.3 (Monotone rigidity). Let A be an n x n matrix over some field. We say that 
A is (r, s)-monotonically rigid if A cannot be written as A = L + S with 

1. L a matrix of rank at most r, 

2. S a matrix with at most s non zeros per column and 

3. The support of S is contained in that of A (that is, Sij ^ implies A^j ^0). 

The following is an immediate corollary of Theorem 14.11 

Corollary 4.4. Let A be annxn matrix with non-negative real entries that is a (q, k, t)-design 
matrix with k > 0(-\/n) and t < 0(1). Then A is {a\f(n) , an)-monotonically rigid for some 
a > 0. For example, one can take A to be the projective plane incidence matrix. 

We observe that this result can be used to derive super linear lower bounds (via Valiant's 
argument) for monotone circuits, which are circuits that can use linear combinations with non 
negative coefficients only. Such lower bounds, however, can be achieved using much simpler 
arguments (in fact, much stronger lower bounds of the form ~ ra 15 ). We are not aware, 
however, of a simple way to construct monotonically rigid matrices. 

5 Proof of Theorem 11.91 

We first prove the general bound for 8 > (later we will analyze the 5 = 1 case). Suppose 
vi,. . . ,v n G C d are a 5-SG configuration. Let V be the n x d matrix whose ith row is the 
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vector Vi. By shifting the points so that «j 7^ for all i G [n] we have 

affine-dim{ui, . . . , u n } = dim{t>i, . . . , u n } — 1 = rank(V) — 1. 

(The difference of 1 between afhne and linear dimension will only matter in the 5 = 1 case.) 

Thus we want to upper-bound the rank of V. To do so, we will construct an m x n design- 
matrix A so that AV = 0. Then we will use the design properties of A to argue that the rank 
of A must be high, which in turn implies that the rank of V must be small. The following 
lemma is implicit in [B DWYlT] and we include its proof sketch here for completeness. 

Lemma 5.1. Let v\, V2, ■ ■ ■ ,v n be a 5-SG configuration. Let V be the n x d matrix whose i 'th 
row is the vector Uj. Then there exists an m x n matrix A such that A is a (3, 3k, 6) -design 
matrix with k = \5(n — l)~\ , every row of A has support of size exactly 3, and such that AV = 0. 

Proof sketch. A result of [Hii73] on the existence of diagonal Latin squares implies that for all 
r > 3 there exists a set T C [r] 3 of r 2 — r triples that satisfies the following properties: 

1. Each triple {ti,t2,ts) G T is of three distinct elements. 

2. For each i G [r] there are exactly 3(r — 1) triples in T containing i as an element. 

3. For every pair i,j G [r] of distinct elements there are at most 6 triples in T which contain 
both i and j as elements. 

Let C denote the set of all special lines in the configuration. Let Lj be a subset of C 
containing lines passing through Vi. For each £ G C let Vn denote the set of points in the 
configuration which lie on the line I. Then \V(\ > 3 and we can assign to it a family of triples 
Tc C Vf satisfying the three properties above. We now construct the matrix A by going over 
all special lines £ £ £ and for each triple t = k) G T( adding as a row of A a vector with 
three non-zero coefficients in positions i,j,k, corresponding to the linear dependency among 
the collinear vectors Vi,Vj,Vk so that we have AV = 0. We now argue that the matrix A 
is a (3, 3k, 6)-design matrix as follows. The number of non-zeros in each row is exactly 3 by 
construction. For each v\, there are at least k = \5(n — 1)] points (other than v{) on special 
lines through Vi . Summing over all of these lines we get that Vi appears in at least 3A: triples and 
so the i'th column of A will contain at least 3k non-zeros. Every distinct pair of points Vi,Vj 
determine a unique line and so, by construction, can appear in at most 6 triples together. □ 



Given this lemma and our rank bounds from the previous section, the proof of Theorem II .9 
follows quite easily. 



Proof of Theorem \1.9l By Corollary 11.41 

n n n 
rank(A) > — — - > — > r^— 

1 + fc 1 + 3r«5(n-l)l 1 + Sn-1 

Now, 



n Yin 12 
To~ = n — > n -. 
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Hence, 

12 

affine-dimjui, «2, • • • , v n } = rank(F) — 1 < n — rank(A) < — . 

o 



□ 



5.1 The case of 5 = 1: Kelly's Theorem 

We now describe how to obtain the tight bound of 2 on the affine-dimension when 6=1. In 
this scenario, every pair of points is on a special line. We start with the following simple claim. 

Claim 5.2. Let A be an m x n matrix so that A is a (q, k, t) design matrix and such that the 
support of each row in A is exactly q. Then m(f\ < \^)t 

Proof. We count the number of pairs of locations in the matrix which are in the same row and 
are both nonzero. Counting once by rows we get that this quantity is equal to fn[t). On the 
other hand, counting by columns (going over all pairs of columns) we get an upper bound of 
(2)2 since two columns intersect in at most t places. □ 

Applying Lemma 15.11 we get an m x n matrix A which is a (3, 3(n — l),6)-design matrix. 
By the above claim we have that m < n(n — 1). Using Theorem 11.31 we get that 

, . n n n(n — 1) 
rank(yl) > — — -. — -r— = j- = — - 1 > n - 4. 

^ I - , 3-2-n(n-l)-6 ]_ i 4 n + 3 

1 ^ n(3(n-l)) 2 n-1 



Hence, 
Thus 
and 



rank(y) < n — rank(A) < 4. 
dim{ui,u 2 , ... ,v n } = rank(y) < 3, 
amne-dim{wi, «2, . . . ,v n } < 2. 



5.2 Proof of high dimensional variant 

Fix some point vq G V. By a normalization w.r.t. vq we mean an afhne transformation 
N : C d ^ C d which first moves vq to zero, then picks a hyperplane H s.t. no point in V 
(after the shift) is parallel to H (i.e., has inner product zero with the orthogonal vector to H) 
and finally multiplies each point (other than zero) by a constant s.t. it is in H. It is easy to 
see (see [BD WYllj ) that, for such a mapping N, we have that Vo, v%, . . . , are dependent iff 
N(v\), . . . , N(vk) are dependent. 

We now prove Theorem 1 1.141 in two parts (corresponding to the two cases of V and V*). We 
denote by f(5, k) the maximum d such that there exists a <5-SG£ configuration of dimensions d. 
We denote by g(5, k) the maximum d such that there exists a <5-SGfc configuration of dimensions 
d. 
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Proof for 8-SG k configurations: The proof is by induction on k. For k = 1 we know f(8, 1) < 
c/8 with c > 1 a universal constant. Suppose k > 1. We separate into two cases. The first 
case is when is a (<5/(2/c))-SGi configuration and we are done using the bound on k = 1. 
In the other case there is some point vq G y* s.t. the size of the set of points on special lines 
through vo is at most 5 /(2k). Let 5 denote the set of points on special lines through vq. Thus 
|5| < 5n/(2k). Let N : C d h-> C d be a normalization w.r.t. t>o- Notice that for points v S 
the image iV(f ) determines u. Similarly, all points on some special line map to the same point 
via N. 

Our goal is to show that V = N(V* \{v }) is a ((1 - 1/ (2k))8)-SG* k _ 1 configuration (after 
eliminating multiplicities from V). This will complete the proof since dim(V*) < dim(V') + 1. 
Indeed, if this is the case we have 

f(5, k) < m&x{2c(k/8), /((l - l/(2k))8, k - 1) + 1}. 

and, by induction, we have f(8,k) < 4c(k/8). 

Fix v[, . . . , v' k _-y G V to be k — 1 independent points (if no such tuple exists then V' is 
trivially 1-SG^_ 1 configuration). Let v\,...,Vk-\ G V* be (necessarily independent) points 
s.t. N(vi) = v[ for i G [k — 1]. Thus, there is a set U C V* of size at least Sn s.t. for 
every u £ U either u G Q.(vo,vi, . . . , t>fc_i) or the /c-flat fl(uo, «i, • • • , ^fc-i, u) contains a point w 
outside fl(vo; • • • > w fc-i) U {^}- 

Let U = U \ S so that iV is invertible on U and 

|C/| > \U\ - \S\ > (1 - l/(2k))5n. 

Suppose u £ U and let u' = N(u). If n G fl(vo, V\, . . . , Wfc-i) then is in fl(t^, . . . , -u^, -J. 
Otherwise, fl(fo, • • • , ffc-i, w) contains a point u; outside B.(vo,vi, . . . , ujfc_i) U {n}. Let 
u;' = N(w). We will show that if/ is (a) contained in the (A; — l)-flat &(v[, . . . , v' k _i, u') 
and (b) is outside Q.(v[, . . . , u^j) U {u'}. Property (a) follows since vo, v\, . . . , Vk-i, u, w are 
dependent and so v[, . . . , v' k _ 1 , u' , w' are also dependent. To show (b) observe first that the 
points v[, . . . , v' k _ 1 ,u' are independent (since Vo, v%, . . . , v k _i,u are independent) and so v! is 
not in Q.(v[, . . . ,v'u_-<). We also need to show that w' ^ u' but this follows from the fact that 
u ^ w and so w' = N(w) ^ N(u) = u' since N is invertible on U and u G U. Since 

\N(U)\ = \U\ > (1 - l/(2k))5n > (1 - \/{2k))5\V'\ 

the proof is complete. □ 

We now prove the second part of Theorem 11.141 

Proof for S-SGk configurations. The proof follows by induction on k (the case k = 1 is given 
by Theorem 1 1.9p . Suppose k > 1. Suppose that dim(V) > #((5, fc). We want to show that there 
exist k independent points vi, . . . ,v k s.t. for at least 1 — 8 fraction of the points w G V we 
have that w is not in fl(fi, . . . ,v k ) and the flat fl(fi, . . . ,v k ,w) is elementary (i.e., does not 
contain any other point). 
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Let k! = g(l, k — 1). Since we are trying to show by induction that g(5, k) < C k /5 for some 
absolute constant C, we can pick C so that g(5, k) > f(S, k' + 1). Therefore, we can find k' + 1 
independent points v\, . . . , Vfe'+i s.t. there is a set U C V of size at least (1 — S)n s.t. for every 
u G U we have that u is not in fl(vi, . . . , Vfc'+i) and the (k' + l)-flat fl(«i, . . . , Vk>+i,u) contains 
only one point, namely u, outside H(v\, . . . , Ufc'+l)- 

We now apply the inductive hypothesis on the set VnH(vi, . . . , Vk'+i) which has dimension 
at least k' = g(l, k—l). This gives us k independent points v[, . . . ,v' k that define an elementary 
(k — l)-flat H(v[, . . . , v' k ). (Saying that V is not l-SGfc_i is the same as saying that it contains 
an elementary (A; — l)-fiat). Joining any of the points u £ U to v[, . . . , v' k gives us an elementary 
A;-flat and so the theorem is proved. □ 



6 Proof of the variation on Freiman's Lemma 

In this section we prove Theorem 11.161 Let A and / : A x A \— > C d be as in the statement of 
the theorem. The proof is divided into two claims. 

Claim 6.1. There exists a subset A' C A with \A\ > Sl(\A\/K) and dim(A') < 0(K 2 ). 

Proof. Let B = A+f A so that for every pair (a, a') £ Ax A there exists some point b G B 
such that /(a, a') = b. Thus, on average, a point b G B has |A| 2 /|i3| pairs of A x A mapping 
to it via the function /. Let B\ be the set of all points in B that have at least |^4| 2 /10|-B| pairs 
that map to it. Let S 1 be the set of pairs of A x A that map to some element of B\. Then 
\S'\ > \A\ 2 /2. Consider B 2 = B\ U A and observe that \B 2 \ < 0(K\A\). Now, each pair in S' 
is on a special line determined by B 2 - Thus, by Corollary 11.111 for a = |S"|/|i?2| 2 j we get that 
there is a subset -Bld C B 2 of dimension at most 0(1 /a) = 0(K 2 ) and size bounded by 

\B LD \ > n(a\B 2 \) > n(\A\ 2 /\B 2 \) > n(\A\/K). 

If \Bld H A\ > \Bld\/2, then take A' = Bld n A and the claim is proved. Otherwise, consider 
the set B' LD = Bld \ A C B\ so that > \Bld\/2- Each point of B' LD has at least 

|A| 2 /10|i?| pairs of A x A that map to it via the function /. For a G A we denote 

M(a) = {a' eA\f(a,a')£B' LD }. 

Then the average of |M(a)| (taken over all a G A) is at least 

1 v-i^.m^ \B'ld\-\A\ 2 



r^r > ^ \M(a)\ > x "\ u \ = n(\A\/K 2 ' 

1 1 a€A iii 

Call a point a £ A a heavy point if 



\M(a)\ > > mM/K 2 ). 



Case (1): Some point a* G A has 



\M(a!)\> K ^ B ^ =n(\A\/K). 
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In this case, consider the set B'[ D = B' LD U {a*}. Clearly this set has dimension at most 
dim(B' LD ) + 1 = 0(l/a) + 1 = 0(K 2 ). Also, the span of B" LD contains the set M(a*). Thus, 
there are Q(A/K) points of A that are contained in a set of dimension 0(K 2 ). This completes 
the proof of this case. 

Case (2): In this case we have that 

, , Nl K\B'\ ■ \A\ 
l M( «>l £ 100IBI 

for all a £ A and, in particular, for all heavy points. Therefore, there must be at least 
heavy points (otherwise, the average of |M(a)| would be too small). Call the set of heavy points 
H, so that \H\ > Q(\A\/K). Pick a\ & H and consider R 1 = span(£^ D U {cti}) n A. Then R x 
contains M(a\), which has size Q(\A\/K 2 ), and has dimension at most dim.(B' LD ) + l = 0(K 2 ). 
If H <£_ Ri, we can pick some ai £ H \ R\ and define R2 = span(B' LD U {cti, 0,2}) n A. The 
dimension of R2 is at most dim(B' LD ) + 2, and its size is at least \R±\ + A \/K 2 ), since 
M(ci2) H M(ai) = 0, or else a2 would be in the span of B' LD U {ai}. Continuing in this manner 
(i.e., picking 03, 04, . . .) for at most K steps or until we run out of elements of H (which has size 
0(|yl|/K)) we obtain a subset A' C A of dimension at most 0(K 2 ) + K = 0{K 2 ) containing 
at least Cl(A/K) elements. This completes the proof of the claim. □ 

Claim 6.2. We have dim(A) < 0(K 2 ). 

Proof. Let A' C A be a subset of size fi(|j4|/-fr) and dimension 0(K 2 ) given by the previous 
claim. Let T be a minimal set in A \ A' for which span(T U A 1 ) contains A. Notice that this 
implies that the points in T are linearly independent and that 

dim(A' U T) = dim(A') + \T\. 



Observe that for every a\ / 02 € T, there do not exist a[ / a' 2 G A' and b £ B such that 

/(ctiX) = /(a 2 ,a' 2 ) = 6. 

This is because otherwise {01,02} C span(yl' U {b}), which means that dim({ai,a2} U A') < 
dim(A') + 1, which violates the properties of T. Therefore, 

\T\ ■ \A'\ < \B\. 

This gives \T\ < 0(K 2 ) and so dim(A) < dim(A') + T = 0(i^T 2 ). □ 
Corollary 6.3. // a set A C M n defines at most K\A\ directions then dim(A) < 0{K 2 ) 

Proof. Notice that the proof of the above theorem works also if the function / is allowed to take 
values in projective space. Since the point at infinity on the line through a, b is the direction 
a — b we get the required consequence. □ 
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